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METHOD AND SYSTEM FOR ANNOTATING A WINDOW SHARED BY REMOTE 

HOSTS 

Field of the Invention 

The present invention relates to a method for annotating an 
5 arbitrary window, for example, one that is shared by remote 

hosts . 

Background of the Invention 

m A computer system, for supporting the collaboration of 

,7 multiple users allows the transmission or sharing of 

lj(§j qualitative knowledge and information that could not be 

iii handled by previously available systems. This system can, 

S for example, add images or graphs to basic data (product 

\Z data, customer data and sales records) , conventionally 

(""": 

□ accumulated in text, so as to share data and thus render it 

15 more useful. Further, the system allows joint development of 

products, or joint studies, with associated companies, and 
for the joint management of projects. In addition, the 
system can rationalize the process through the sharing of 
information with vendors and/or suppliers. Furthermore, when 
20 the range for the exchange of information is expanded to 

include the Internet, upon the receipt of an inquiry from a 
remote customer, for example, a salesperson can output the 
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requested product information to the browser of the customer 
and provide a supplemental explanation, or can, when 
appropriate, take an order. All things being equal, it is 
therefore anticipated that this system will contribute to an 
increase in mobile offices and in stay-at-home working. 

One application for promoting collaboration is the product 
Web Collaboration, by the present applicant { IBM Corp.), that 
has as its primary purpose the sharing of HTML documents for 
web servers. Additional, similar products are NetMeeting, by 
Microsoft, for the primary purpose of sharing applications 
running on a variety of PCs, and Freelance Team Screen Show, 
by Lotus, which has as its main purpose the synchronous 
browsing of a screen show by multiple PCs. Web Collaboration, 
for example, uses JavaScript to write a process for a 
predetermined event by taking the form of an input screen 
prepared using HTML, and performs upon the occurrence of an 
event during an input operation. When this application is 
used for an automatic contract machine installed in a bank, 
the contents of the input screen related to bank procedures 
and to the state of an operation by a customer can be shared 
by remote hosts. Therefore, the degree of understanding 
established between the customer and the bank can be 
increased, and the smooth communication of data can be 
expected. 
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While Web Collaboration uses an annotation function as a tool 
for collaboration between remote hosts, the annotation 
function can also be used to write additional data at a 
specific location on a screen to draw the attention of 
5 another person, and to provide a supplementary explanation. 

According to this annotation function, when remote hosts 
collaborate, with a screen and an application being shared, 
the hosts can converse with each other while highlighting 
specific points on the screen. With Web Collaboration, an 
10 annotation is effected using a method that provides for the 

writing of data in the plug- in window of a web browser. When 
fll the object of an annotation is expanded to take in more than 

L J* the web browser, and the annotation is employed for content 

U] (Shockwave or an Applet) using a plug- in window, the object 

15 to be annotated can be expanded. In addition, when the 

contents of an annotation can be customized, a more effective 
^ annotation can be obtained. 

u 

Software, such as NetMeeting, which transmits the screen 
image of an application to another person to further advance 

20 the progress of a collaboration, has a whiteboard function in 

addition to the application sharing function. The whiteboard 
function is a painting tool to which the image sharing 
function has been added, and according to this function, an 
image can be directly drawn on a current web page, so that in 

25 real time, the image drawn on the canvas of the tool can be 
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shared with another person. With this function, the capture 
of the image of a desktop or of a window can also be 
transmitted to another person to request the entry of 
additional writing. However, with the whiteboard function, 
5 an annotation can not be written to a local application 

employed by a user, and therefore, the whiteboard function is 
not a satisfactory annotation function. 

Furthermore, in Japanese Unexamined Patent publication No. 

Hei 10-254669, a technique is disclosed for displaying an 
Iffi applet as a map on a browser, and for displaying thereon, as 

py accompanying information, a moving image icon representing an 
L 2 object such as a specific, current location. However, 

Ul according to the technique disclosed in this publication, an 

i3 application is one that is accompanied only by the applet of 

l^J a browser, and the application is therefore limited. 

Further, merely one part of an application at a local user is 
Q annotated, and it is not taught in the publication that an 

I.. .Su 

1 annotation mark, for example, can be moved by a manipulation 

performed by an operator. 

20 Summary of the Invention 

To resolve the above shortcomings, it is one object of the 
present invention to provide a collaboration tool for remote 
hosts to annotate an arbitrary application window. 
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It is another object of the present invention to enable the 
use of a moving picture (a line, a circle, an arrow or a 
mascot) as an annotation object, and to customize an 
annotation object for each system. 

5 It is an additional object of the present invention to 

annotate moving picture applications. 

It is a further object of the present invention to rearrange 
annotations and to add annotations to areas separated from 
i.g application windows. 

ID 1 The present invention uses an annotation object as the 

Mi 

LR topmost (TOPMOST) window that can be arbitrarily shaped to 

perform an annotation function for drawing the attention of 

;;;;; another person by adding writing to various applications. At 

HJ this time, as a collaboration tool used by remote hosts, an 

annotation can be performed for an arbitrary window from a 

r ' IS Java application. That is, according to one aspect of the 
present invention, an annotation method for collaboration by 
interconnected hosts comprises the steps of: annotating an 
application window that is displayed on a display unit; 
20 forming or selecting an annotation object that is constructed 

separately from an application that constructs the 
application window; determining a display position for the 
annotation object and displaying the annotation object 
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thereat; and transmitting to the interconnected hosts a 
request event for the sharing of the annotation object, so 
that the annotation object can be shared by the hosts. 

The annotation object is displayed in a window having a 
TOPMOST attribute so that the window is always displayed in 
the foreground. Since the annotation function can be 
employed without any special changes being made, this 
configuration is preferable regardless of whether an 
application to be annotated is possessed by a local user. 
The annotation object is displayed by forming a window from 
which a portion not required for the annotation is removed. 
Thus, the application to be annotated can be protected from 
being completely covered by the annotation object. 

Further, the annotation object is an object possessing 
mobility. This arrangement is superior as an annotation 
function that will draw the attention of a customer. More 
specifically, for example, an Image object of the Java as an 
annotation object in an arbitrary shape can be employed based 
on an image (a moving picture) handled by Java. 
In addition, the annotation method further comprises the step 
of receiving from the interconnected hosts the images 
displayed in the application windows when images displayed on 
desktops by the hosts differ. 
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According to another aspect of the invention, a method for 
additional writing to an application window displayed on a 
desktop comprises the steps of: selecting an object for 
additional writing to the application window; and displaying 
a window, from which an unnecessary portion has been removed, 
for the object by employing a window having a TOPMOST 
attribute to display the object in the foreground screen on 
the desktop. More specifically, a set of rectangles is 
formed of pixels for which the occupation rate is not zero, 
and a region is generated using a region function, so that 
the shape of the window can be set so as not to include a 
transparent portion. 

According to an additional aspect, a computer apparatus 
employing the present invention comprises: a display unit, 
for displaying an application window; an annotation object 
definition file, for storing an annotation object that is 
formed without depending on an application that has 
constructed the application window; a window shaping unit, 
for shaping a window for displaying the stored annotation 
object; an object selector, for selecting an annotation 
object that is additionally written in the application 
window; and a location selector, for designating a location, 
for the display of the selected annotation object, that 
corresponds to a predetermined position in the application 
window, wherein the display unit displays an annotation 
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object window shaped by the window shaping unit at the 
location designated in the application window. 

The window shaping unit detects a transparent attribute of an 
object, which is information for the annotation object, and 
5 permits a predetermined image that is positioned at the 

predetermined location in the application window to be seen 
through. 

_ The computer apparatus further comprises: a sharing request 

ylj transmitter for transmitting an annotation object sharing 

1|Q| request to a computer apparatus that is connected thereto. 

Thus, collaborating remote hosts can share information and 
can use the same annotation. 

An automatic contract machine that employs the present 
j;^ invention comprises: display means, for displaying 

liSI predetermined windows, including an application window 

: :j 
jilts; 

employed when jointly preparing a contract with a customer; 
and reception means, for receiving, from a connected host 
apparatus, a request event for the sharing of an object that 
is used to draw the attention of a customer to the 
20 application window, wherein the display means processes the 

object based on the sharing request event, and displays an 
object window in the application window. 
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In the process performed for the object by the display means, 
a location for the display of the object is designated, and a 
window designated for the object is shaped using a region 
function. Thus, an annotation object having an arbitrary 
shape can be employed by providing a desired image for an 
Image object prepared with Java. 

According to a further aspect of the invention, a system for 
collaboration by a first host and a second host is provided, 
the first host comprising means for forming or selecting an 
annotation object to be used when annotating an application 
window displayed by the second host, and means for 
transmitting, to the second host, a request event for the 
sharing of the annotation object that is formed or selected, 
and the second host comprising means for receiving from the 
first host, the request event for the sharing of the 
annotation, means for processing the annotation object based 
on the sharing request event, and means for displaying with 
the application window a window concerning the annotation 
object that has been processed. 



The first host further comprises: means for displaying a 
predetermined application window and the selected annotation 
object, wherein the same application window as an application 
window displayed by the second host, is displayed by the 
means for displaying, and wherein the annotation object is 
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displayed on the application window. Further, the first host 
comprises: acquisition request means for requesting the 
second host the acquisition of a currently displayed 
application window. 

According to a still further aspect of the invention, a 
computer- readable storage medium for storing a program that 
permits a computer to perform: a process for forming or 
selecting an annotation object to draw attention to a 
specific portion of a desktop image displayed on a desktop; a 
process for determining a display position for the annotation 
object; and a process for transmitting, to a connected host, 
a sharing request event that includes information concerning 
the annotation object and the determined display position. A 
storage medium such as a CD-ROM can be employed for this 
purpose . 

According to one more aspect of the invention, a computer 
program product for controlling the display of a computer 
comprises: a first sub-routine, stored on a storage medium, 
for permitting the computer to perform a process for 
designating a location on a desktop, whereat an annotation 
object window is displayed, for the writing and the display 
on the desktop of an additional application window; and a 
second sub-routine, also stored on the storage medium, for 
permitting the computer to perform a process for displaying 
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on the desktop screen, using a window having an attribute 
that ensures the window will be displayed in the foreground 
of the screen, an annotation object window in which an 
unnecessary portion is transparent, 

5 According to yet one more aspect of the present invention, a 

program transmission apparatus comprises: storage means for 
storing a program that permits a computer to perform a 
process for forming or selecting an annotation object to draw 
attention to a specific portion of a desktop image displayed 
l|§ on a desktop, a process for determining a display position 
m for the annotation object, and a process for transmitting, to 
a connected host, a sharing request event that includes 
hp. information concerning the annotation object and the 

ls determined display position; and transmission means for 

reading the program from the storage means and for 
Nj transmitting the program. 

[" ;!;; : 

Ui 

According to yet another aspect of the present invention, a 
program transmission apparatus comprises: storage means for 
storing a program that permits a computer to perform a 
20 process for designating a location on a desktop, whereat an 

annotation object window is displayed, for the writing and 
the display on the desktop of an additional application 
window, and a process for displaying on the desktop screen, 
using a window having an attribute that ensures the window 
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will be displayed in the foreground of the screen, an 
annotation object window in which an unnecessary portion is 
transparent; and transmission means for reading the program 
from the storage means and for transmitting the program. 

Brief Description of the Drawings 

The present invention will now be described in detail during 
the course of an explanation of the preferred embodiment, 
given while referring to the accompanying drawings, in which: 

Fig. 1 is a schematic diagram showing the configuration of an 
annotation window according to one embodiment of the present 
invention; 

Figs. 2A, 2B and 2C are diagrams showing the effects obtained 
by shaping a window; 

Fig. 3 is a diagram showing an example window shaping method; 

Fig. 4 is a functional block diagram explaining the general 
configuration of a control system according to the 
embodiment ; 

Fig. 5 is a flowchart explaining the processing performed by 
the control system according to the embodiment; 
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Figs. 6A and 6B are diagrams showing an example format for an 
annotation object definition file and an example definition 
file; 

Fig, 7 is a flowchart showing the event reception processing 
5 performed by an event receiver; and 

Fig. 8 is a diagram showing an example display for an 
annotation object according to the embodiment. 

ry Detailed Description of the Invention 

f;, : 

y] Before beginning a detailed explanation of the functions of 
lj) the system according to the present invention, the annotation 

;S method used for this embodiment will be described. As 
previously remarked, the annotation function is one that, for 
Q example, provides for the insertion of additional writing 

into a specific location on a screen in order to furnish a 
15 supplemental explanation which draws the attention of another 

person. This function can be further defined as one that 
exaggerates a specific portion of a screen for a customer. 
According to the method provided by this embodiment, 
effective lines, circles, ellipses, rectangles, arrows 
20 (pointing in all directions) and mascots (all of which are 

hereinafter referred to as annotation objects) are prepared 
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using animated GIF (Graphics Interchange Format) , and are so 
arranged at locations that draw the user's attention. 

Fig. 1 is a schematic diagram for explaining the arrangement 
of an annotation window in accordance with one embodiment . 
5 In Fig. 1, an application window 202 is displayed on a 

desktop 201, and an annotation object 203 is displayed at a 
location to be annotated in the application window 202. 

Since an application to be annotated may be independent of an 
application that provides an annotation tool, the annotation 
object cannot be directly written to the annotation target 

: ': application. Therefore, in this embodiment, a window having 

a TOPMOST attribute is employed to display the annotation 

I object 203. According to the TOPMOST attribute, the window 

is always displayed in the foreground of the desktop 201. 

ff When the window of the TOPMOST attribute is employed, the 

□ annotation function can be used regardless of the 

manufacturer of the target application. 

As is described above, since the annotation object 203 must 
be displayed in front of the application window 2 02, the 
20 TOPMOST attribute is provided for the window, using Win3 2 (a 

32-bit API built into Windows 95/98/NT/2000) by a JNI (Java 
Native Interface: an API (Application Program Interface) for 
employing, for a program written in the Java Language, a 



14 



JP920000246US1 



native code program that is developed in another language) . 

The substance of the annotation object 203 is defined by 
inheriting the Java base class. The base class is a Window 
object having a Panel object that is a standard Java AWT 
(Abstract Windowing Toolkit) component. When an arbitrary AWT 
component is combined thereon, the original annotation object 
can be obtained. 

Since the window formed using the Java Window class is 
normally a rectangular window, it is difficult to say this 
window is appropriate for indicating a specific portion of an 
application. For example, when a desired portion to which 
attention is to be drawn is to be displayed with a circle or 
pointed at by an arrow, it would be meaningless for the 
circle or the arrow to be so placed in the window that it 
covers the application to be annotated. Thus, in the 
embodiment, Win32 is used for the JNI in order to form a 
window from which a portion unnecessary for annotation is 
removed. To do this, a function that will be described later 
is used to form a designated window shape indicated by a 
region object. 

Figs. 2A, 2B and 2C show the effects obtained by shaping a 
window. In Fig. 2A, no annotation is performed for an 
application window 211. In Fig. 2B, a common window 212 with 
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no transparent attribute is used for annotation. In Fig. 2C, 
an annotation object window 213 is shaped in accordance with 
one embodiment of the present invention is employed. In Fig. 
2B, an image file comprising the common window 212 with no 
transparent attribute is employed as an annotation object. In 
Fig. 2B the application window 211 is covered by the 
rectangular area. In Fig. 2C, the annotation object window 
213 is shaped like an ellipse, and all the application window 
211 other than the portion under the elliptical outline form 
is displayed. Further, since the annotation object window 
213 has the TOPMOST attribute, i.e., the WS_EX_TOPMOST 
attribute for Windows, a window is provided that is always 
displayed as the topmost. As a result, this window can be 
displayed in different application programs (Microsoft Word, 
Internet explorer, etc.) running other Window processes. 

Fig. 3 shows an example window shaping method. In this 
example, the transparent attribute of the annotation object 
that constitutes the information for the annotation object is 
detected, and the non- transparent portion is divided into 
multiple rectangles 221. Then, the Window region objects are 
formed and unified to obtain a single region object. In Fig. 
3, the shaping of an ellipse is shown. When this region 
object is transmitted to Windows, the shape of the form for 
the window can be prepared. 
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As used in this embodiment, a method based on the alpha plane 
of a Java Image object is employed. Specifically, a Java 
Image object is examined beginning at the end to find pixels 
whose occupation rate (alpha value) is not zero (pixels 
5 having an alpha value of 0 are transparent) , and a set of 

rectangles 221 is obtained. Then, the region objects are 
formed by the region function of Win32, and the shape of the 
window is set for one having no transparent portion. In this 
embodiment, a method is used whereby a region object is 
ljO represented as a set of rectangular regions having an 

up arbitrary shape. This is because the affect produced by the 

m use of a division algorithm to reduce the number of 

rectangles 221 is taken into account. 

111 

is Since a desired image need only be provided for the Image 

1|S| object, an annotation object having an arbitrary shape can be 
^ employed. Since the Java Image object is also compatible 
Q with animated GIF (Graphics Interchange Format) , a moving 
annotation object can easily be designed using a tool such as 
a web animator. Furthermore, when the method for generating 
20 an image of the Image object from a program is employed, a 

more specialized annotation can be prepared. 

When the annotation function of this embodiment is performed 
for an application on the screen of another party to a 
25 collaboration, the host of the other party must be notified 
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the annotation operation is being used. The Java RMI can be 
employed to effect the notification of the display and the 
rearrangement performed for an annotation object and an event 
process. Further, since whether or not each annotation . 
5 object should be displayed on the screen of the other party 

can be set as needed, the interface for displaying the 
annotation object can be prepared after the positioning and 
the alteration of the size of the object have been performed. 
In addition, an annotation object operation using a mouse can 
10 also be reported to the other party in real time. 

i|| Furthermore, when the window layouts on the screens of the 
m hosts that are parties to a collaboration differ, one host 

,7 employs a tool that receives the screen image from the other 

yi host and designates the annotation location. 

An explanation will now be given for the general 
HJ configuration of a control system that carries out the 

Q annotation method and for the processing that is performed. 
1 Fig. 4 explains the general arrangement of the control system 
of this embodiment. The control system in Fig. 4 constitutes 
20 a function with which each host (PC terminal, etc.) that is a 

party to a collaboration is equipped. For example, in a 
financial facility such as a bank, this system is installed 
in an automatic contract machine provided for customer use 
and in a server provided for bank use. It should be noted, 
25 however, that the automatic contract machine provided for 
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customer use may only display an annotation object window, 
and that the functions for forming, selecting and 
transmitting an annotation object window may be removed. 

In Fig. 4, upon the receipt of an entry from an input device 
5 11, such as a keyboard or a pointing device such as a mouse, 

a tool window 12 selects a function for a display 
instruction, a deletion instruction, a movement instruction, 
a sharing instruction or a display instruction when desktop 
images differ. Based on the selection made by the tool 
lffi window 12, a flow controller 13 assumes control of the 

m selected function. In an annotation object generator 14, a 

definition file reader 15 reads an annotation object 
y] definition file 17, obtains information concerning the 

Eg annotation object information, and transmits the information 

l|S to a window shaping unit 16. The window shaping unit 16 

! aaife detects the transparent attribute of the annotation object 

Q included in the information, divides the non- transparent 

portion into multiple rectangles 221 (as shown in Fig. 3) , 
forms region objects for the Windows, and collects them into 
20 one region object, and outputs the region object to a display 

unit 31. 

In a display instruction unit 18, an object selector 19 
selects a necessary annotation object from multiple objects 
that are prepared in advance by the annotation object 
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generator 14. A position designator 20 determines a display 
position for the annotation object selected by the object 
selector 19 and outputs the display position to the display 
unit 31. Upon the receipt of an instruction from the 
position designator 20, the display unit 31 displays an 
annotation object window 32. Then, a deletion instruction 
unit 21 issues an instruction to the display unit 31 to 
delete the annotation object window 32, while a movement 
instruction unit 22 notifies the display unit 31 of a 
position after a movement has been performed and changes the 
display position of the annotation object window 32. 

The sharing instruction unit 23 includes a sharing request 
transmitter 24 for transmitting a sharing request event to a 
connected host, which, for example, is a PC based on TCP/IP, 
and, for a financial facility, corresponds to an automatic 
contract machine or a server installed for the use of a bank. 
When the annotation object is shared by the hosts in response 
to a sharing request from the sharing request transmitter 24, 
the name, position, display and deletion functions for the 
annotation object are synchronized. Additionally, the 
sharing request event that is issued includes information for 
the name and the position of the annotation object, and for 
the display or deletion of the object. 

An object selector 26 in the display instruction unit 25 
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selects a necessary annotation object from among multiple 
objects prepared in advance by the nnotation object generator 
14. Further, an image acquisition request transmitter 27 
transmits a request event to obtain, from a connected host, a 
5 desktop image, and upon receipt of the desktop image an image 

reception and display unit 2 8 displays it in a window 3 0 for 
designating the location of the connected host. Then, based 
on the information displayed in the window 30, a position 
designator 29 determines the display position for the 
10 ; annotation object and the display unit 31 displays the 

uQ annotation object window 3 2 at the determined position. 

■7 An event receiver 41 receives an event from the connected 
Ul host, and an event flow controller 42 controls a succeeding 
ls process in accordance with the type of received event, either 

l]|5 a sharing request event or a desktop image acquisition 
request event. In a desktop image acquisition request 
Q processor 43, an image acquisition unit 44 obtains a 

jsws 

currently displayed desktop image, and an image transmitter 
45 transmits an image file to the connected host. A sharing 
20 request processor 46 then processes a target annotation 

object in accordance with the information included in the 
sharing request . 

At a setup time, such as upon the activation of the system, a 
setting unit 4 7 is called to write an initial value (a path 
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for the annotation object definition file 17) to an initial 
setting file 48. 

Fig. 5 is a flowchart explaining the processing performed by 
the control system in accordance with the embodiment . 
First, during the initial process, an initial value (a path 
for the annotation object definition file 17) is read from 
the initial setting file 48, and is transmitted to the flow 
controller 13 (step 101) . Then the annotation object 
generator 14 reads the annotation object definition file 17, 
obtains the annotation object information, transmits it to 
the window shaping unit 16, and generates the annotation 
object (step 102) . Then, when desktop images differ, the 
tool window 12 selects a function for the display 
instruction, the deletion instruction, the movement 
instruction, the sharing instruction or the display 
instruction (step 103) . 

Fig. 6A shows an example format for the annotation object 
definition file 17. Fig. 6B shows an example of an 
annotation object definition file 17. In the format example 
in Fig. 6A, information such as the set name and the number 
of images is stored. In Fig. 6B the annotation object 
definition file 17, stores information such as a name for 
identifying an ellipse (ELLIPSE) or a line (LINE) and a 
moving picture comprising a plurality of structures. As is 
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shown, the annotation object definition file 17 can be 
customized for each company, i.e., for each customer who 
operates the collaboration system. For example, in a bank 
system, a character used by each bank can be defined as an 
annotation object, and a more effective annotation process 
can be performed. 

Referring again to Fig. 5, when the display instruction 
function is selected (step 104) , program control is shifted 
to routine A. When the deletion instruction function is 
selected (step 105), program control is shifted to routine B. 
Then, when the movement instruction function is selected 
(step 106), program control is shifted to routine C, and when 
the sharing instruction function is selected (step 107) , 
program control is shifted to routine D. Thereafter, when 
the display instruction function that is used when desktop 
images differ is selected (step 108), program control is 
shifted to routine E, and when routines A to E are 
terminated, program control returns from F to step 103 and 
the process for the selected function is performed. 

In routine A, i.e., when the display instruction function is 
selected (step 111) from multiple annotation objects that are 
prepared in advance by the annotation object generator 14, 
the object selector 19 of the display instruction unit 18 
selects a necessary annotation object (step 112) . The tool 
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window 12 may be employed for this object selection, or the 
API may select the object from a program. Thereafter, the 
display position of the annotation object is designated by 
the position designator 2 0 (step 113) , and the annotation 
object display process is performed to display the annotation 
object window 32 on the display unit 31 (step 114) . Then, 
when the display instruction process is terminated, program 
control returns to F to wait for the selection of a new 
function. 

In routine B, i.e., when the deletion instruction function is 
selected (step 121) , the deletion instruction unit 21 
performs the annotation object deletion process (step 122) . 
During this process, a deletion instruction is issued to the 
display unit 31, and the annotation object window 32, which 
is displayed once, is deleted. Then, when the deletion 
process is terminated, program control returns to F to wait 
for the selection of a new function. 

In routine C, i.e., when the movement instruction function is 
selected (step 131) , the movement instruction unit 22 
performs the annotation object movement process (step 132) . 
During this process, in order to move the annotation object, 
which is displayed once, the movement instruction unit 22 
notifies the display unit 31 of the location following the 
movement, and changes the display position of the annotation 
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object window 32. Thereafter, program control returns to F 
and waits for the selection of a new function. 

In routine D, i.e., when the sharing instruction function is 
selected (step 141) , the sharing process is performed by the 
5 sharing instruction unit 23 (step 142) . During this process, 

the sharing request transmitter 24 transmits a sharing 
request event to a connected host. More specifically, on the 
reception terminal side, the annotation object (annotation 
content) is arranged using the input device 11, such as a 
lj£ mouse, and an instruction is issued for the object to also be 

fu displayed on the sharing party side. Then, when the 

J annotation object is also displayed by the sharing party and 

y! is shared by the hosts, the positioning, the displaying and 

I iff 

the deletion of the annotation object are synchronized. 
l!l Thereafter, program control returns to F and waits for the 

M* selection of a new function. 

t : 
-saw 

In routine E, i.e., when the display instruction function 
used when desktop images differ is selected (step 151) , the 
object selector 26 of the display instruction unit 25, which 
20 is operated when the desktop images differ, selects a 

necessary annotation object from among those prepared in 
advance by the annotation object generator 14 (step 152) . 
Thereafter, the image acquisition transmitter 27 and the 
image reception/ display unit 2 8 obtain a desktop image from 
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the connected host (step 153), and the position designator 29 
designates the display position (step 154) . Then, the 
display unit 31 displays the annotation object window 32 
(step 155) . Thereafter, program control returns to E and 
waits for the selection of a new function. As is described 
above, when the desktop images of the hosts differ, i.e., 
when the desktop display contents at the reception and the 
sharing party's terminals differ, the annotation object can 
be displayed only on the sharing party side. In order to 
position the annotation object on the sharing party's screen, 
the desktop image on the sharing party's side can be captured 
and be converted into a GIF file or a JPEG file, which can 
then be transmitted to the reception terminal. Then, the 
received annotation object can be displayed in the window at 
the reception terminal and the positioning determined, so 
that the annotation object can be displayed on the sharing 
party's side. 

Fig. 7 shows the event reception processing performed by the 
event receiver 41 for an event transmitted by a connected 
host. First, the function is selected as an event type (step 
171) , which, as previously described, is a sharing request 
event or a desktop image acquisition request event. A check 
is then performed to determine whether the event is a desktop 
image acquisition request (step 172), and when the event type 
is a desktop image acquisition request, it is accepted (step 
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173) . Following this, the image acquisition unit 44 obtains 
a desktop image (step 174) that the image transmitter 45 
transmits (step 175) . The processing is thereafter 

terminated. 

If the event type at step 172 is not a desktop image 
acquisition request, a check is performed to determine 
whether the event is a sharing request (step 176) . If the 
event type is not a sharing request, program control returns 
to step 171 and waits for selection of a function. If the 
event type is a sharing request, the sharing request 
processor 46 performs the sharing request process (step 177) 
and the sharing process for handling the annotation object 
designated by the information included in the sharing request 
event (step 178) . Thereafter, the process sequence is 
terminated. 

Fig. 8 shows an example display for an annotation object 
according to the embodiment. Annotation objects are 
displayed in an application window 3 01, which is, for 
example, an input screen for procedures provided by an 
automatic contract machine. A tool window 302, comprising 
four buttons, is provided in the application window 3 01. The 
tool window 3 02 includes a line annotation object button 3 03 
which uses underlining for annotation, an ellipse annotation 
object button 3 04 which uses an elliptic shape for 
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annotation, a sharing button 3 05 which is used for sharing an 
annotation object with another party, and a deletion button 
3 06 for deleting the annotation object. When a user calls an 
annotation object window using the line annotation object 
5 button 3 03 and the ellipse annotation object button 3 04 to 

issue a movement instruction, the user can, for example, use 
the mouse to perform the drag and drop operation. Further, 
in addition to issuing an instruction for moving from the 
tool window 3 02, the user can also, for example, operate the 
mouse or depress the tab key to detect an event wherein the 
v3 object is brought into focus or a focus is lost, and can 
fy display the event on the upper left of the object that is 
; ! brought into focus. This method can be used as an auxiliary 
I'Jj application for a manipulation by the operator. 

s!!3 

l£ In the example display in Fig. 8, an elliptic annotation 

!^ object window 3 07 and an annotation object window 3 08 (having 
CI the form of a character mark) are shown. Because of the 

TOPMOST attribute, the annotation object windows 3 07 and 3 08 
are displayed in the application window 301. A portion of 

2 0 each of these windows not required for annotation has been 

removed and includes a transparent attribute. In addition, 
the elliptic annotation object window 3 07 is displayed using 
a moving picture that dynamically draws, for example, a red 
and orange colored ellipse. Other annotation objects can be 

25 used, such as a mascot symbolizing the bank depicted as a 
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moving picture. 

Several display methods can be employed. For example, after 
the image file for a moving picture has been displayed for a 
specific period of time, the image file for a static image 
can be displayed. According to this method, when multiple 
annotation objects are arranged in a window, attention will 
be drawn to the annotation object that is last displayed. An 
image file for a moving picture that continues to be 
displayed is an effective annotation method for continuously 
attracting attention to the annotation object. 

As is described above in detail, according to the embodiment, 
the annotation object, which is the TOPMOST window and that 
can be formed using an arbitrary shape, can be employed to 
annotate various applications that include moving pictures. 
That is, instead of writing directly into a sharing window, 
the annotation object is independently formed, and thus, an 
object window can be written into a moving picture (e.g., an 
Applet, animated GIF, Shockwave, etc.). By using this 
function, annotation objects can also be easily rearranged. 

Since a standard AWT component is used as a base, the 
original annotation object can be easily generated, and can 
be customized for each system. Further, while an Image 
object is employed, an annotation object having an arbitrary 
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shape can be employed based on an image (a moving picture) 
handled by Java. In a collaboration entered into by remote 
hosts sharing a screen or an application, a situation exists 
where the hosts converse while designating a specific point 
on the screen. According to this embodiment, a smooth 
communication is ensured. 

A program for performing the annotation process explained in 
this embodiment is stored on a storage medium and is provided 
for sale. This storage medium can be one, such as a CD-ROM, 
on which software for performing the above process is stored. 
Further, when downloading this program across a network, a 
storage medium for a program transmission apparatus, or a 
storage medium such as a hard disk for storing the downloaded 
program, can also be used. The program that is stored on a 
storage medium can be read by input means, such as a CD-ROM 
driver. Further, a program transmission apparatus need only 
include, for example, interface means (transmission means) 
for, upon the receipt of a downloading request from a PC 
terminal connected to the Internet, transmitting a program 
that carries out the embodiment. 

As is described above, according to the present invention, 
annotation can be performed for windows provided by various 
applications . When the present invention is applied to a 
system used for collaboration, explanations can be smoothly 
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